Skip to content

Fix gemini and codex for opencode - #79

Closed
mpkrass7 wants to merge 11 commits into
mainfrom
marshall/llm-option-fix
Closed

Fix gemini and codex for opencode#79
mpkrass7 wants to merge 11 commits into
mainfrom
marshall/llm-option-fix

Conversation

@mpkrass7

@mpkrass7 mpkrass7 commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

Make the stupid models work

The terminal session strips DATABRICKS_TOKEN from the env for security,
so `{env:DATABRICKS_TOKEN}` in opencode.json interpolated to empty
string. Opencode only falls back to auth.json when `options.apiKey`
is undefined (provider.ts L1683), so the empty value silently
overrode the freshly-rotated auth.json token. GPT/Codex models hit
the gateway direct (no proxy fallback) and returned "Credential was
not sent". Drop apiKey from both providers' options and rely on
auth.json, which cli_auth._update_opencode already keeps in sync.

Also add Claude Haiku 4.5 for parity and drop the duplicate
databricks-gemini-2-5-pro entries in both config blocks.

Co-authored-by: Isaac
@mpkrass7 mpkrass7 changed the title fix(opencode): read auth from auth.json, not stripped env var Fix gemini and codex for opencode Jun 7, 2026
mpkrass7 added 10 commits June 7, 2026 10:31
Opencode's auth loader expects {"type": "api", "key": ...} and silently
drops entries that don't decode. Our previous {"api_key": ...} format
got dropped, leaving provider.key undefined → @ai-sdk/openai threw
"API key missing" once options.apiKey was removed.

Write the correct schema in setup_opencode.py and rotate the `key`
field (not `api_key`) in cli_auth._update_opencode. Updates the
related tests too. Also trims oversized explainer comments.

Co-authored-by: Isaac
GPT-5.x reasoning models send `reasoningSummary` (Responses API
field) on every request. The content-filter proxy forwards to the
chat-completions endpoint, which rejects unknown params. Add it to
the proxy's strip-list alongside stream_options.

Also swap databricks-gpt-5-1-codex-max (not served in stablebox)
for databricks-gpt-5-2-codex, which is.

Co-authored-by: Isaac
gpt-5.4-mini and friends reject reasoning_effort when combined with
function tools on /v1/chat/completions. Strip it the same way we
strip reasoningSummary. Full reasoning is still available via the
direct databricks-openai route which hits /v1/responses.

Co-authored-by: Isaac
Previous strip was blanket and would silently kill Claude's thinking
budget (gateway translates reasoning_effort → Anthropic thinking
params). Restrict the reasoning-field strip to GPT models only;
Claude and Gemini keep it.

Co-authored-by: Isaac
Co-authored-by: Isaac
Gemini's function declaration schema only accepts a narrow subset of
JSON Schema. Add exclusiveMinimum/exclusiveMaximum/multipleOf/
uniqueItems to the strip list alongside the existing \$schema/
additionalProperties.

Co-authored-by: Isaac
Databricks gateway returns Gemini deltas as Anthropic-style content
block arrays ([{type:"text", text:"...", thoughtSignature:"..."}]),
but opencode's openai-compatible Zod schema requires content to be a
plain string. Add _flatten_content_blocks() to the proxy so both
streaming deltas and non-streaming messages get collapsed before
opencode parses them. thoughtSignature and other non-text blocks are
dropped.

Also register databricks-gemini-3-5-flash and
databricks-gemini-3-1-flash-lite in both gateway-mode and fallback
opencode configs.

Co-authored-by: Isaac
Both branches were 95% identical — only the databricks provider's
display name differed, and gateway-mode added an extra
databricks-openai provider. Pull the models dicts up to module
constants and conditionally append the openai provider. Adding a new
model is now a one-line edit instead of two.

Net -117 lines, same generated output.

Co-authored-by: Isaac
…viders

Three related changes squashed together:

1. Workspace introspection (setup_opencode.py)
   At setup time, query Databricks serving-endpoints and models.dev's
   bundled databricks catalog. Mark catalog entries the workspace
   doesn't serve as enabled:false; surface workspace-only chat
   endpoints (Claude Opus 4.8, GPT-5.5 Pro, Llama 4 Maverick, Qwen3,
   etc.) with their gateway display name. Embeddings filtered out by
   task=="llm/v1/chat". Fail-soft on network errors.

2. Llama integer-schema fix (content_filter_proxy.py)
   Llama 4 Maverick's validator rejects "minimum" on integer types.
   Walk the schema and drop minimum/maximum when parent type is
   "integer". String minLength/maxLength and number bounds preserved.

3. Provider whitelist (setup_opencode.py)
   Add enabled_providers config so only databricks (proxy) and
   databricks-openai (direct) show in the picker. Hides everything
   opencode auto-loads from models.dev (google, anthropic, openai,
   etc.) which polluted the picker with no-credentials entries like
   Gemini 3 Flash Preview.

Co-authored-by: Isaac
@mpkrass7
mpkrass7 force-pushed the marshall/llm-option-fix branch from 7dbe0a9 to 04061d4 Compare June 8, 2026 16:14
dgokeeffe added a commit that referenced this pull request Aug 5, 2026
… for Gemini/GPT (#119)

Cherry-picks the still-relevant parts of #79 (mpkrass7) onto current main.
Merging that branch as-is would have reverted a lot: it predates main's
SP-OAuth token resolution, proxy tracing, mtime-invalidated token cache, the
opus-4-8 catalog with 1M context limits, and the enterprise_config npm_env
wiring. Only the fixes main is actually missing are taken.

## auth.json was the wrong shape

opencode stores credentials as a map of provider-id -> credential, where the
credential is a discriminated union on `type`. The API-key variant keeps the
secret in `key`:

    export class Api extends Schema.Class<Api>("ApiAuth")({
        type: Schema.Literal("api"),
        key: Schema.String,
        metadata: Schema.optional(Schema.Record(Schema.String, Schema.String)),
    }) {}

    const _Info = Schema.Union([Oauth, Api, WellKnown])
        .annotate({ discriminator: "type", identifier: "Auth" })

(opencode, packages/opencode/src/auth/index.ts)

`api_key` is not a field opencode recognises. Both sides wrote it:
setup_opencode.py produced `{"databricks": {"api_key": ...}}`, and
cli_auth._update_opencode() rotated `api_key` every 10 minutes — so the
credential was unloadable and rotation updated a key nothing reads.

It went unnoticed because the two were *consistently* wrong (every unit test
agreed with them — the old tests asserted `api_key` explicitly), and because
OpenCode routes through the content-filter proxy, which injects a fresh bearer
token per request and masks the broken credential at runtime.

The shape now lives once in `utils.opencode_api_credential()` /
`is_opencode_api_credential()`, shared by writer and rotator. That, rather than
fixing both call sites, is what stops it drifting again — same reasoning as the
existing `workspace_sync_dest()` helper. Rotation is also now scoped to
`type == "api"`, so an `oauth` or `wellknown` credential can't have a PAT
written into it.

## Tool-less requests were skipping sanitisation entirely

`sanitize_tool_schemas()` early-returned when a request had no `tools`, so the
top-level cleanup below it — `stream_options`, `$schema`, and now the reasoning
keys — never ran on a plain chat turn. Found by a test written for the new GPT
stripping, which failed until the early return came out. Pre-existing on main.

## Proxy compatibility fixes

- Strip `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`, `uniqueItems`.
  Gemini 400s the whole request on these rather than ignoring them, so an
  unstripped key makes the tool unusable, not merely unvalidated.
- Drop `minimum`/`maximum` on `type: integer` only. Kept for `number`, since
  over-stripping loses real constraints.
- Strip `reasoning_effort` / `reasoningSummary` for GPT-routed models only,
  matched on the model id. Stripping globally would silently downgrade output
  on models that support reasoning.
- Flatten Anthropic-style `content` block arrays to a plain string in both the
  non-streaming `message` and the streaming `delta`. OpenAI-shaped clients
  expect a string and render the raw array otherwise. Absent `content` is left
  absent rather than set to "", so a tool-call-only message isn't turned into
  an empty assistant turn.

## Not taken

The setup_opencode.py model-catalog rewrite (main's is newer) and the
requirements.txt `pydantic-core` pin (main already has 2.46.4 after #110).

## Verification

535 tests pass. New coverage: tests/test_opencode_auth_schema.py holds the
writer/rotator contract that was missing — whatever the writer emits, the real
rotator must be able to rotate, with 0600 preserved — plus the schema-stripping,
GPT-scoping, flattening, and tool-less-request cases. Confirmed the auth and
early-return tests fail without their fixes.

Note: the Gemini/GPT fixes are latent while app.yaml disables Codex and Gemini
(no compatible gateway endpoints). The auth.json fix is not latent — OpenCode
is enabled.

Co-authored-by: Marshall Krassenstein <mpkrass7@users.noreply.github.com>
@dgokeeffe

Copy link
Copy Markdown
Collaborator

Landed via #119 (merged), with credit to you in the commit trailer — thank you for finding these.

I cherry-picked rather than merging this branch because main has moved a long way in the same files since it was opened: taking it as-is would have reverted the SP-OAuth token resolution, the proxy tracing spans, the mtime-invalidated token cache in content_filter_proxy.py, and the opus-4-8 catalog with 1M context limits plus enterprise_config.npm_env wiring in setup_opencode.py.

What went in:

  • The auth.json tagged-union fix — and you were right, this was a genuine bug. I verified against opencode's own packages/opencode/src/auth/index.ts: the credential is a union discriminated on type with the secret in key, so api_key was unloadable. It was wrong in both setup_opencode.py and cli_auth._update_opencode(), which is why rotation had been quietly updating a field nothing reads. I moved the shape into utils.opencode_api_credential() so the writer and rotator share one definition, and scoped rotation to type == "api" so oauth/wellknown credentials aren't clobbered.
  • All four proxy fixes: the extra Gemini schema keys, the integer-only minimum/maximum drop, GPT-scoped reasoning_effort/reasoningSummary stripping, and content-block flattening in both message and delta.

One extra thing your change surfaced: sanitize_tool_schemas() early-returned for requests without tools, so the top-level cleanup (stream_options, $schema) never ran on plain chat turns. Your version dropped that early return; a test I wrote for the GPT stripping failed until I did the same. Pre-existing on main, now fixed.

Not taken: the setup_opencode.py model-catalog rewrite (main's is newer) and the pydantic-core pin (main is already on 2.46.4).

Also added tests/test_opencode_auth_schema.py, which asserts the writer/rotator contract that was missing — the reason the mismatch survived was that the old tests asserted api_key explicitly, so they agreed with the bug.

@dgokeeffe dgokeeffe closed this Aug 5, 2026
dgokeeffe pushed a commit that referenced this pull request Aug 5, 2026
* fix: extend session linger to 24 hours (closes #76)

Sessions now survive for up to 24 hours of inactivity before cleanup
reaps them. Active sessions with heartbeats live indefinitely — the
timeout only applies to abandoned sessions. Cleanup interval bumped
from 60s to 15min since frequent sweeps are unnecessary with a 24h
window.

* fix: close slave FD after Popen and set 32 MB upload limit

- Close slave_fd in parent after Popen to prevent FD leak (fixes #78)
- Set MAX_CONTENT_LENGTH to 32 MB aligned with Claude Code's 30 MB
  file limit (fixes #79)
dgokeeffe added a commit that referenced this pull request Aug 5, 2026
… for Gemini/GPT (#119)

Cherry-picks the still-relevant parts of #79 (mpkrass7) onto current main.
Merging that branch as-is would have reverted a lot: it predates main's
SP-OAuth token resolution, proxy tracing, mtime-invalidated token cache, the
opus-4-8 catalog with 1M context limits, and the enterprise_config npm_env
wiring. Only the fixes main is actually missing are taken.

## auth.json was the wrong shape

opencode stores credentials as a map of provider-id -> credential, where the
credential is a discriminated union on `type`. The API-key variant keeps the
secret in `key`:

    export class Api extends Schema.Class<Api>("ApiAuth")({
        type: Schema.Literal("api"),
        key: Schema.String,
        metadata: Schema.optional(Schema.Record(Schema.String, Schema.String)),
    }) {}

    const _Info = Schema.Union([Oauth, Api, WellKnown])
        .annotate({ discriminator: "type", identifier: "Auth" })

(opencode, packages/opencode/src/auth/index.ts)

`api_key` is not a field opencode recognises. Both sides wrote it:
setup_opencode.py produced `{"databricks": {"api_key": ...}}`, and
cli_auth._update_opencode() rotated `api_key` every 10 minutes — so the
credential was unloadable and rotation updated a key nothing reads.

It went unnoticed because the two were *consistently* wrong (every unit test
agreed with them — the old tests asserted `api_key` explicitly), and because
OpenCode routes through the content-filter proxy, which injects a fresh bearer
token per request and masks the broken credential at runtime.

The shape now lives once in `utils.opencode_api_credential()` /
`is_opencode_api_credential()`, shared by writer and rotator. That, rather than
fixing both call sites, is what stops it drifting again — same reasoning as the
existing `workspace_sync_dest()` helper. Rotation is also now scoped to
`type == "api"`, so an `oauth` or `wellknown` credential can't have a PAT
written into it.

## Tool-less requests were skipping sanitisation entirely

`sanitize_tool_schemas()` early-returned when a request had no `tools`, so the
top-level cleanup below it — `stream_options`, `$schema`, and now the reasoning
keys — never ran on a plain chat turn. Found by a test written for the new GPT
stripping, which failed until the early return came out. Pre-existing on main.

## Proxy compatibility fixes

- Strip `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`, `uniqueItems`.
  Gemini 400s the whole request on these rather than ignoring them, so an
  unstripped key makes the tool unusable, not merely unvalidated.
- Drop `minimum`/`maximum` on `type: integer` only. Kept for `number`, since
  over-stripping loses real constraints.
- Strip `reasoning_effort` / `reasoningSummary` for GPT-routed models only,
  matched on the model id. Stripping globally would silently downgrade output
  on models that support reasoning.
- Flatten Anthropic-style `content` block arrays to a plain string in both the
  non-streaming `message` and the streaming `delta`. OpenAI-shaped clients
  expect a string and render the raw array otherwise. Absent `content` is left
  absent rather than set to "", so a tool-call-only message isn't turned into
  an empty assistant turn.

## Not taken

The setup_opencode.py model-catalog rewrite (main's is newer) and the
requirements.txt `pydantic-core` pin (main already has 2.46.4 after #110).

## Verification

535 tests pass. New coverage: tests/test_opencode_auth_schema.py holds the
writer/rotator contract that was missing — whatever the writer emits, the real
rotator must be able to rotate, with 0600 preserved — plus the schema-stripping,
GPT-scoping, flattening, and tool-less-request cases. Confirmed the auth and
early-return tests fail without their fixes.

Note: the Gemini/GPT fixes are latent while app.yaml disables Codex and Gemini
(no compatible gateway endpoints). The auth.json fix is not latent — OpenCode
is enabled.

Co-authored-by: Marshall Krassenstein <mpkrass7@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants